Image processing apparatus which performs image processing on photographic subject

ABSTRACT

First, the position and the size of a target photographic subject such as the face of a person in an original image acquired by image capturing is acquired, and then mask information indicating a relation between a position to be subjected to predetermined image processing in the original image and the strength of the predetermined image processing is generated based on the acquired position and size. Subsequently, a new image is generated by predetermined image processing of a different strength being performed on each position in the original image by use of the generated mask information. In the generation of the new image, for example, one of the original image and a processed image acquired by the predetermined image processing being performed on the original image is combined with the other image by use of transmissivity indicated by the mask information for each pixel.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2015-010793, filed Jan. 23,2015, the entire contents of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and animage processing method.

2. Description of the Related Art

Conventionally, as a type of image processing that is performed in adigital camera and the like, soft focus processing is known in which adefocusing effect similar to that occurred due to the chromaticaberration and spherical aberration of a dedicated lens is added to anoriginal image so as to create a soft atmosphere. As a technique relatedto this soft focus processing, for example, Japanese Patent ApplicationLaid-Open (Kokai) Publication No. 2000-069277 discloses a technique inwhich the face of a person is detected from an original image andprocessing strength is controlled based on the size of the face so asnot to perform the processing uniformly regardless of the contents ofthe image.

SUMMARY OF THE INVENTION

In accordance with one aspect of the present invention, there isprovided an image processing apparatus comprising a CPU that isconfigured to acquire a position and a size of a target photographicsubject in an original image acquired by image capturing; generate maskinformation indicating a relation between (i) a position to be subjectedto predetermined image processing in the original image and (ii)strength of the predetermined image processing, based on the acquiredposition and the acquired size of the target photographic subject; andgenerate a new image by performing predetermined image processing of adifferent strength on each position in the original image by using thegenerated mask information.

In accordance with another aspect of the present invention, there isprovided an image processing apparatus comprising: a CPU that isconfigured to identify, as a target photographic subject, a photographicsubject for which focal adjustment has been made and which is shown inan original image acquired by image capturing; acquire focal informationof the identified photographic subject; generate mask informationindicating a relation between (i) a position to be subjected topredetermined image processing in the original image and (ii) strengthof the predetermined image processing, based on the acquired focalinformation; and generate a new image by performing predetermined imageprocessing of a different strength on each position in the originalimage by using the generated mask information.

In accordance with another aspect of the present invention, there isprovided an image processing method comprising: acquiring a position anda size of a target photographic subject in an original image acquired byimage capturing; generating mask information indicating a relationbetween (i) a position to be subjected to predetermined image processingin the original image and (ii) strength of the predetermined imageprocessing, based on the acquired position and the acquired size of thetarget photographic subject; and generating a new image by performingpredetermined image processing of a different strength on each positionin the original image by using the generated mask information.

The above and further objects and novel features of the presentinvention will more fully appear from the following detailed descriptionwhen the same is read in conjunction with the accompanying drawings. Itis to be expressly understood, however, that the drawings are for thepurpose of illustration only and are not intended as a definition of thelimits of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be more deeply understood by the detaileddescription below being considered together with the following drawings.

FIG. 1 is a block diagram showing a digital camera that is common toeach embodiment of the present invention:

FIG. 2 is a flowchart of image capture processing in a first embodiment;

FIG. 3A is a diagram showing a first mask image, and FIG. 3D is adiagram showing a second mask image;

FIG. 4 is a diagram showing an example of an image generated by theimage capture processing in the first embodiment;

FIG. 5A and FIG. 5B are diagrams showing a method for generating a thirdmask image;

FIG. 6 is a diagram showing another form of a second mask image; and

FIG. 7 is a flowchart of image capture processing in a secondembodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will hereinafter be described withreference to the drawings.

First Embodiment

First, a first embodiment of the present invention is described. FIG. 1is a block diagram showing the electrical structure of a digital camera1 that is an example of an image processing apparatus according to thefirst embodiment and a second embodiment described later.

This digital camera 1 includes a control section 2, a lens section 3, animaging section 4, a display section 5, an image storage section 6, aprogram storage section 7, a face detection section 8, an operationsection 9, and a power supply section 10, as shown in FIG. 1.

The control section 2 includes a CPU (Central Processing twit), itsperipheral circuits, an internal working memory such as a RAM (RandomAccess Memory), and an image processing circuit. This control section 2controls each section of the digital camera 1, and performs variousimage processing described later.

The lens section 3 is constituted by a lens group including a focus lensand a zoom lens, a motor that drives the lens group, an aperture, and anactuator that adjusts the opening of the aperture by driving theaperture to be opened or closed.

The imaging section 4 includes an image sensor of a CCD (Charge CoupledDevice) or CMOS (Complementary Metal Oxide Semiconductor) type, and anAFE (Analog Front End) that performs predetermined analog processing onan imaging signal outputted from the image sensor and converts it into adigital signal. When the digital camera 1 is in an imaging mode, theimaging section 4 sequentially captures a photographic subject at apredetermined frame rate via the lens group of the lens section 3, andsupplies the image data (imaging data) of the photographic subject tothe control section 2.

The image data supplied to the control section 2 is then supplied to thedisplay section 5, and displayed on the display section 5 as a live viewimage.

Image data supplied to the control section 2 from the imaging section 4in image capturing performed in response to an imaging executioninstruction given using the operation section 9 described later issubjected to development processing including gamma correction, whitebalance adjustment, generation of RGB color component data for eachpixel, YUV conversion, and the like, and compressed in the JPEG (JointPhotographic Experts Group) format. Subsequently, various attributioninformation is added to the compressed image data. Then, the image datais stored in the image storage section 6 as a still image file thatmeets the Exif (Exchangeable Image File Format) standard.

The image storage section 6 includes, for example, a flash memoryembedded in the digital camera 1, various memory cards detachablyattached to the digital camera 1, and a card interface that enables theinput and output of data to and from a memory card.

When the digital camera 1 is in a playback mode, image data stored as astill image file in the image storage section 6 is read out by thecontrol section 2 as necessary, and expanded. Then, the image data issupplied to the display section 5 and displayed on the screen.

The display section 5 includes a color liquid crystal display panel anda display driving circuit that drives the color liquid crystal displaypanel in accordance with image data and the like supplied from thecontrol section 2. As described above, in the imaging mode, the displaysection 5 displays a subject image by live view display. In the playbackmode, the display section 5 displays captured images constituted byimage data stored as still image files in the image storage section 6.

The program storage section 7 includes, for example, a non-volatilememory such as a ROM (Read Only Memory) or a flash memory where storeddata can be rewritten any time. In the program storage section 7, aprogram for causing the control section 2 to perform processingdescribed later is stored in advance.

Also, the program storage section 7 has stored thereon a program forcausing the control section 2 to perform AE (Auto Exposure) control, AF(Auto Focus) control, AWB (Auto White Balance) control, and the like,and various data, such as data constituting a program chart indicating acombination of shutter speed, ISO speed, and an aperture value that areused in AE control. Note that the AF control by the control section 2 iscontrol using the so-called contrast detection method where the imagesensor of the imaging section 4 is used as an AF sensor.

Moreover, the program storage section 7 has stored thereon a program forcausing the control section 2 to perform image capture processingdescribed later and mask data 101 that is used in the image captureprocessing. The mask data 101 is gray scale image data representing afirst mask image M1 shown in FIG. 3A.

This first mask image M1 corresponds to one of four images acquired by asecond mask image M2 shown in FIG. 3B being divided into four images inthe peripheral direction. The second mask image M2 is an image having anoval gradation area on a black background, in which the luminosity ofeach pixel is maximum at the center and gradually decreases toward theperiphery.

Here, in the present embodiment, the second mask image 142 correspondsto basic information of the present invention which is used to generatea third mask image 143 described later, and the shape (oval shape) ofthe gradation area is a basic shape of a processing area wherepredetermined image processing in the present invention is performed.Also, change in the luminosity of each pixel in the gradation areaindicates change in the strength of the predetermined image processingof the present invention, or more specifically, indicates the fact thatthe processing strength gradually increases toward the periphery sidefrom the center side.

The face detection section 8 includes an image processing circuit thatperforms face detection processing for detecting the face portion of anarbitrary person from an image captured by the imaging section 4, amemory having stored thereon various types of data to be used for theface detection processing, a working memory for storage, and the like,and supplies area information indicating an area corresponding to theface portion to the control section 2.

The face detection processing by the face detection section 8 iswell-known versatile processing in which eyes, a nose, and a mouthhaving a predetermined positional relationship are searched for by usingan image recognition technique such as binarization, contour extraction,and pattern matching, and a rectangular area (hereinafter referred to as“face area”) where they are present is identified and detected as a faceportion.

The operation section 9 includes a plurality of operation switches, suchas a power button, a shutter button, a mode change switch for switchingbetween the imaging mode and the playback mode, a zoom lever, and acontrol button that is used for selecting and setting various functions.The operation statuses of the operation switches in the operationsection 9 are continuously monitored by the control section 2.

The power supply section 10 includes a rechargeable battery, a DC/DCconverter, and the like, and supplies electric power required for eachoperation to each section of the digital camera 1.

In the present embodiment, the digital camera 1 has a soft focusfunction as a function that can be used in the imaging mode. This softfocus function is a function by which predetermined processing includingsoft focus processing equivalent to conventional soft focus processingis automatically performed on an image acquired in image capturing andthe processed image is stored as a definitive captured image.

In the imaging mode of the digital camera 1, when the soft focusfunction has been set by the user, the control section 2 performs imagecapture processing shown in the flowchart of FIG. 2 in accordance withthe program stored in the program storage section 7.

That is, when the imaging mode is set the control section 2 starts thecapturing of a photographic subject image by the imaging section 4 andlive view display by the display section 5, enters an imaging standbystate, and maintains this state until an instruction to perform imagingis given by the user (NO at Step SA1), as shown in FIG. 2. Note that,although not shown, the control section 2 in the imaging standby stateperforms AF control.

Then, when an instruction to perform imaging is given by the user (YESat Step SA1), the control section 2 performs normal developmentprocessing on the data of a captured image acquired by the imagingsection 4 and thereby generates a normal developed image so that theimage can be stored (Step SA2).

Subsequently, the control section 2 performs soft focus processing onthe normal developed image generated as described above, and therebygenerates a soft focus image, that is, a processed image to which adefocusing effect similar to that occurred due to the chromaticaberration and spherical aberration of a dedicated lens has been added(Step SA3). Note that this soft focus processing is processing using awell-known image processing technology including one or a plurality ofprocessing steps.

Next, the control section 2 controls the face detection section 8 toperform face detection processing with the normal developed image as aprocessing target (Step SA4).

Here, when the face of an arbitrary person is not detected from thenormal developed image by the face detection section 8 (NO at Step SA5),the control section 2 takes the soft focus image as a captured image,that is, a definitive image to be stored, performs compressionprocessing and the addition of attribution information, and stores theimage in the image storage section 6 as a still image file (Step SA6).Then, the control section 2 ends the image capture processing.

Conversely, when the face of an arbitrary person is detected from thenormal developed image (YES at Step SA5), that is, when a normaldeveloped image G1 such as that shown in FIG. 4 has been generated bythe processing at Step SA2 and a soft focus image G2 such as that shownin FIG. 4 has been generated by the processing at Step SA3, the controlsection 2 performs the processing of Step SA7 and the following steps.

Here, the processing at Step SA7 and the following steps are brieflyexplained prior to the explanation of its concrete details. That is, thecontrol section 2 generates the above-described second mask image 142shown in FIG. 3B using the first mask image M1 shown in FIG. 3A, andgenerates the third mask image M3 shown in FIG. 4 using the second maskimage M2. Note that the third mask image M3 is a gray scale image aswith the first mask image M1, and its image size (number of pixels) isthe same as that of the normal developed image G1.

Then, the control section 2 sets, for each pixel of the normal developedimage G1, transparency based on the luminosity of each pixel of thethird mask image 143, or in other words, a transparency level at whichan image on a background side when the normal developed image G1 issuperimposed thereon is shown through the normal developed image G1.Subsequently, the control section 2 combines the normal developed imageG1 and the soft focus image G2 with the soft focus image G2 as abackground, and thereby generates a composite image G3 shown in FIG. 4.That is, by well-known alpha blending, the normal developed image G1 andthe soft focus image G2 that serves as a background are combined,whereby the composite image G3 is generated. Note that, in FIG. 4, anarea (effect reduction area described later) corresponding to thegradation area of the third mask image M3 is indicated by an oval Xshown with a broken line on the composite image G3 for descriptivepurposes.

Hereafter, the processing at Step SA7 and the following steps isprecisely described. First, the control section 2 acquires the centerportion and the size of an image of the face area in the normaldeveloped image G1 detected by the face detection section 8 (Step SA7).FIG. 5A is a diagram showing the normal developed image G1, the facearea. W, and its center O. In the face processing at Step SA7, thecontrol section 2 acquires the longitudinal size A of the face area W asthe size of the face area W, and acquires the coordinates of the centerO of the face area W as its position in the image.

Next, the control section 2 calculates the size and the center portionof the second mask image M2 for the generation of the third mask imageM3 shown in FIG. 4, based on the acquired data (the coordinates of thesize A and the center O), (Step SA8). Here, the size of the second maskimage M2 to be calculated is the longitudinal size C of a portioncorresponding to the second mask image M2 in the third mask image M3that is generated later, and the center portion is the coordinates ofthe center P of the second mask image M2 in the third mask image M3. Inprocessing at Step SA8, the control section 2 multiplies thelongitudinal size A of the face area W by a predetermined number, andthereby calculates the longitudinal size C. FIG. 5B shows an examplewhere the predetermined number is 2.

Also, in the processing at Step SA8, the control section 2 calculatescoordinates that are acquired by moving the coordinates of the center Oof the face area W in the normal developed image G1 upward by apredetermined percentage (n %) of the longitudinal size A, as thecoordinates of the center P of the second mask image M2 in the thirdmask image M3. For example, when the coordinates of the center of theface area W are (x0, y0), the movement amount B in the y-axis directionis A×n %, and the control section 2 acquires (x0, y0-B) as thecoordinates of the center P of the second mask image M2. FIG. SB is anexample where the predetermined percentage is 40%.

Here, the above-described predetermined number and the predeterminedpercentage are values determined in consideration of the differencebetween the face area W identified by the face detection processing atStep SA4 and the actual face portion of the person excluding the hairportion in the normal developed image G1. Specifically, they are valuesdetermined based on an empirical rule.

Then, the control section 2 processes the first mask image M1 shown inFIG. 3A based on the calculated data (the longitudinal size C of thesecond mask image M2 and the coordinates of the center P), and therebygenerates the third mask image M3 shown in FIG. 4 which is a mask imagefor image processing (Step SA9).

Specifically, the control section 2 first vertically and horizontallyinverts the first mask image M1, and thereby generates three new maskimages. Subsequently, the control section 2 connects them with the firstmask image M1, and thereby generates the second mask image M2 shown inFIG. 3B. Next, the control section 2 enlarges the generated second maskimage M2 such that its longitudinal size becomes equal to theabove-described longitudinal size C, and performs smoothing processingon the enlarged second mask image M2 so as to ensure that the luminancevalue of each pixel changes smoothly.

Then, the control section 2 combines the smoothed second mask image M2and a black image of a gray scale having the same pixel size as thenormal developed image G1, with the black image as a background and withthe center P being arranged at the coordinates calculated as describedabove, and then overwrites the second mask image M2, whereby the thirdmask image M3 is generated. Here, a surplus portion of the black imageprotruded into the area of the enlarged and smoothed second mask imageM2 is trimmed.

After generating the third mask image M3, the control section 2 combinesthe normal developed image G1 and the soft focus image G2 by alphablending using the third mask image M3, and thereby generates thecomposite image G3 (Step SA10).

Here, the control section 2 sets transmissivity according to theluminosity of each pixel of the third mask, image M3 for each pixel ofthe normal developed image G1 as described above. Specifically, thecontrol section 2 sets, for each pixel of the normal developed image G1,a transmissivity of 0% to 100% according to the luminance value of acorresponding pixel of the third mask image M3. That is, when theluminosity of a corresponding pixel is high, a low transmissivity isset. Specifically, the control section 2 sets the lowest transmissivity(0%: completely opaque) for the pixel of the center P corresponding tothe second mask image M2, sets transmissivity that gradually increasestoward the periphery of the gradation area for other pixels, and setsthe highest transmissivity (100%: completely transparent) for the otherareas in the third mask image M3 excluding the gradation area.

Note that, when performing the alpha blending of the normal developedimage G1 and the soft focus image G2, the RGB value of each pixel iscalculated using an alpha value (1-0) according to the above-describedtransmissivity.

As a result, the composite image G3 is acquired in which a defocusingeffect on the face portion of the person by the soft focus processinghas been decreased as compared to the other portions while ensuring animage effect on the entire image by the soft focusing, and the level ofthe defocusing effect is gradually increasing from the center sidetoward the periphery side, or in other words, the reduction level of thedefocusing effect is gradually decreasing toward the periphery, as shownin FIG. 4.

Then, the control section 2 performs compression processing and theaddition of attribution information on the generated composite image G3that is a definitive image to be stored, and stores it in the imagestorage section 6 as a still image file (Step SA11). Then, the controlsection 2 ends the image capture processing.

Thus, in the present embodiment, when a main photographic subject, thatis, a target photographic subject is a person in image capturing usingthe soft focus function, the strength of the processing on the faceportion of the person, that is, the defocusing effect on the faceportion is suitably controlled regardless of the strength of the softfocus processing, whereby a captured image that is suitable as an imageto be recorded can be acquired.

In addition, in the definitive captured image (composite image G3)acquired as the image to be recorded, the reduction level of thedefocusing effect by the soft focus processing is gradually decreasingfrom substantially the center of the face portion of the person towardthe periphery, and therefore a boundary between defocusing levels doesnot appear between the face portion of the person and the other areas,whereby a natural image effect can be ensured in the captured image.

Also, the position of the face area W acquired by the general-purposeface detection processing is determined as the position of the faceportion of the person in the standard developed image G1. Accordingly, aprocessing load required to acquire the position is light.

Moreover, in the generation of the third mask image M3, the center P ofthe gradation area in the third mask image M3 is shifted to a position(the upper side in the present embodiment) different from the center Oof the face area W as described above. Accordingly, even if a well-knowntechnique is used in which the face of a person is detected based on itsarea, the position of the gradation area, that is, the position of thearea where the defocusing effect by the soft focus processing is reduced(hereinafter referred to as “effect reduction area”) coincides with theactual face portion of the person. As a result, a favorable capturedimage is acquired as a definitive captured image while reducing aprocessing load required to acquire the position of the face portion ofthe person.

Furthermore, in the generation of the third mask image M3, the secondmask image M2 is enlarged in accordance with the size of the face areaW, so that the effect reduction area (gradation area) is enlarged tohave a size by which the face portion of the person is unfailinglycovered. By this enlargement as well, a favorable captured image isacquired as a definitive captured image.

Still further, when a main photographic subject is an object other thana person such as scenery, a captured image (soft focus image) can beacquired on which an image effect has been given on the entire area bysoft focusing as with conventional techniques.

Yet still further, the third mask image M3 is generated only when a mainphotographic subject is a person. Accordingly, when a main photographicsubject is an object other than a person, unnecessary processing forgenerating the third mask image M3 can be omitted. On the other hand,the third mask image M3 is generated based on the first mask image M1stored in advance. That is, since the basic shape that is basicinformation regarding the above-described effect reduction area and thechange content of the reduction level of the processing effect arestored in advance as the first mask image M1, the processing required togenerate the third mask image M3 can be reduced.

Also, since the first mask image M1 is equivalent to an image acquiredby the second mask image M2 being divided into four images, the basicinformation regarding the third mask image M3 can be efficiently stored.

Here, the processing of the present embodiment has been described on thepremise that the normal developed image G1 mainly includes only oneface. However, in the generation of the third mask image M3 when thefaces of a plurality of persons are present in the normal developedimage G1, a plurality of effect reduction areas based on the positionand size of each face are acquired. In this case, when there is anoverlapping area where the effect reduction areas are overlapping witheach other, the reduction level of the processing effect in theoverlapping area is set to correspond to that of the effect reductionarea having the highest reduction level.

Also, in the present embodiment, the position and the size of an effectreduction area to be set in the third mask image M3 are adjustedaccording to the position and the size of the face of a person (the sizeof the face area W). However, in addition to this, the shape of theeffect reduction area may be adjusted according to the shape of theface. Specifically, the ellipticity of the effect reduction area (ovalarea) may be adjusted based on the ratio of the longitudinal size andthe transverse size of the face area W, that is, based on whether theface is a slender face or a plump face. In this case, a more accurateeffect reduction area can be acquired by the third mask image M3. Notethat the difference of the faces of persons herein includes not onlyshape differences among individuals in literal terms but also shapedifferences due to differences of imaging directions.

Moreover, the present embodiment has been described on the premise thatthe direction of the face of a person and the direction of an imagecoincide with each other. However, in a case where a person lying downis imaged, the person may have turned his or her head to a side, andtherefore the vertical direction of the face may not coincide with thatof a captured image. Therefore, in the present invention, it ispreferable that the orientation of the face of a person, that is, therotation direction and the rotation amount of the face in an image isacquired in the above-described face detection processing, and thesecond mask image M2 is rotated based on the orientation of the face ofthe person when the third mask image M3 is generated. By thisconfiguration as well, a more accurate effect reduction area can beacquired by the third mask image M3.

In the case where the shape and the orientation of an effect reductionarea to be set in the third mask image M3 are adjusted based on theratio of the longitudinal size and the transverse size of the face areaW or the orientation of the face of a person, a plurality of mask imageswhose shapes and orientations of effect reduction areas are differentfrom each other are stored in advance as the first mask image M1, andthe third mask image M3 is generated by these masks being usedselectively, whereby the processing required to generate the third maskimage M3 can be reduced.

Furthermore, in the present embodiment, the second mask image M2 shownin FIG. 3B is used to generate the third mask image M3. However, thepresent invention is not limited thereto, and another mask image M21having a gradation area in which the position Q of the pixel having themaximum luminance has been decentered within the oval area can be usedto generate the third mask image M3, as shown in FIG. 6.

In this case as well, by the reduction level being changed to begradually decreased from the center side toward the periphery side, aboundary between defocusing levels does not appear between the faceportion and the other areas, whereby a natural image effect can beensured in the captured image.

Also, in this case, in the generation of the third mask image M3, theabove-described pixel position Q is arranged to be positioned at thecenter P of the second mask image M2 shown in FIG. 5B, whereby theeffect reduction area can coincide with the actual face portion of theperson. In addition, by using a mask image acquired by theabove-described other mask M21 being divided horizontally in the diagramas the first mask image M1, the basic information regarding the thirdmask image M3 can be efficiently stored.

In the present embodiment, the shape of an effect reduction area to beset in the third mask image M3 is oval. However, the shape of thiseffect reduction area may be changed as necessary.

That is, the shape of the effect reduction area may be changed asnecessary in accordance with a target photographic subject or an imagingmode for performing image capturing according to a target photographicsubject. For example, when a target to be in the effect reduction areais an object other than the face of a person, such as when the target isan animal such as a cat, the shape may be wide oval. When the target isa flower, the shape may be true circle. When the target is a building,the shape may be rectangle.

Second Embodiment

Next, a second embodiment of the present invention is described. Thesecond embodiment is different from the first embodiment in that aphotographic subject focused on by AF control in image capturing istaken as a main photographic subject, that is, a target photographicsubject. That is, in this embodiment, in the program storage section 7of the digital camera 1 having the structure shown in FIG. 1, thefollowing data and program have been stored in addition to the variousdata described above.

Specifically, in the program storage section 7 in this embodiment, adistance acquisition table has been stored which indicates a relationbetween an adjusted position of the lens group in the lens section 3 anda focusing distance (a distance to a focused photographic subject).Also, the program storage section 7 has stored thereon a program forcausing the control section 2 to perform AF control by so-calledcontinuous AF by which focusing is continuously performed on arelatively active photographic subject in an image, and a program forcausing the control section 2 to perform image capture processing shownin FIG. 7 in the imaging mode when the soft focus function is set by theuser.

Here, the difference between the image capturing processing in thisembodiment and the image capturing processing in the first embodiment isdescribed first. In the present embodiment the control section 2 doesnot perform face detection in image capturing. The control section 2 ofthe present embodiment generates the third mask image based on adistance to a photographic subject focused on by AF control (hereinafterreferred to as “focusing distance”) and a focal position, and thengenerates a definitive captured image (image to be recorded) using thethird mask image.

Next, the processing by the control section 2 of the present embodimentis precisely described using the flowchart in FIG. 7. As shown in thedrawing, processing from Step S51 to Step S53 is the same as that of thefirst embodiment.

That is, when an instruction to start imaging is given by the user in animaging standby state during which live view display and AF control areperformed (YES at Step S51), the control section 2 controls the imagingsection 4 to capture an image to be recorded, performs normaldevelopment processing on the data of the captured image, and generatesa normal developed image (Step SB2). Subsequently, the control section 2performs soft focus processing on the entire normal developed imagegenerated as described above, and thereby generates a soft focus image(Step S53).

Next, the control section 2 acquires a focusing distance at the time ofthe imaging start instruction, and a focal position, that is, thecoordinates of the focused area in the image (Step S54). Note that thefocusing distance is acquired from the above-described distanceacquisition table based on the adjusted position of the lens group atthe time of the imaging start instruction.

Next, the control section 2 acquires the size and the center position ofthe second mask image M2 (refer to FIG. 5) that is a reference maskimage, based on the acquired data (the focusing distance and the focalposition) (Step SB5).

The size of the second mask image M2 to be acquired herein is equal tothe longitudinal size of a portion of the third mask image (not shown)to be generated later which corresponds to the second mask image M2, asin the case of the first embodiment. Also, the longitudinal size to beacquired herein changes according to the focusing distance and the focallength of the lens section 3, or in other words, the zoom magnification.That is, this longitudinal size becomes smaller as the focusing distancebecomes longer or the focal length of the lens section 3 becomes longer.The control section 2 parameterizes the focusing distance and the focallength of the lens section 3, and acquires the longitudinal size bycalculation using a predetermined function by which the longitudinalsize becomes smaller as the focusing distance becomes longer or thefocal length of the lens section 3 becomes longer. That is the controlsection 2 adjusts the size of the portion of the third mask imagecorresponding to the second mask image M2 such that the size becomessmaller as the focusing distance becomes longer or the focal length ofthe lens section 3 becomes longer.

Then, the control section 2 processes the first mask image M1 shown inFIG. 3A based on the acquired data (the longitudinal size of the secondmask image M2 and the focal position), and thereby generates the thirdmask image that is a mask image for image processing (Step SB6).

That is, after generating the second mask image M2 from the first maskimage M1 as in the case of the first embodiment, the control section 2enlarges the size thereof such that the longitudinal size becomes equalto the previously acquired longitudinal size, and performs smoothingprocessing on the enlarged second mask image M2. Then, the controlsection 2 combines the smoothed second mask image M2 and a black imageof a gray scale having the same pixel size as the normal developedimage, with the center P being arranged at the previously calculatedfocal position, and overwrites the second mask image M2, whereby thethird mask image M3 is generated. Note that in this embodiment as well,a surplus portion of the black image protruded into the area of theenlarged image is trimmed, as shown in FIG. 5B.

Then, the control section 2 combines the soft focus image generated asStep SB3 and the normal developed image by alpha blending using thegenerated third mask image M3, and thereby generates a composite image(Step SB7). Note that the details of the processing are the same as thatof the first embodiment.

As a result, the composite image is acquired in which a defocusingeffect has not been exerted on the area of the photographic subjectfocused on at the time of the image capturing while ensuring an imageeffect on the entire image by the soft focusing.

Then, the control section 2 takes the generated composite image as acaptured image (an image to be recorded), performs compressionprocessing and the addition of attribution information thereon, andstores it in the image storage section 6 as a still image file (StepSB8). Then, the control section 2 ends the image capture processing.

Thus, in this embodiment as well, when a main photographic subjectfocused on by a photographer is a person in image capturing using thesoft focus function, a captured image can be acquired in which adefocusing effect has not been exerted on the face portion of the personwhile ensuring an image effect on the entire image by the soft focusing.Also, when a main photographic subject is an object other than a person,a captured image can be acquired in which a defocusing effect has notbeen exerted on the photographic subject portion. That is, in thepresent embodiment as well, the strength of the processing on a mainphotographic subject can be suitably controlled regardless of thestrength of the soft focus processing, whereby a favorable capturedimage can be acquired.

Also, in this embodiment as well, the second mask image M2 is an imagehaving an oval gradation area, and the reduction level of an effectreduction area set in the third mask image M3 gradually decreases fromsubstantially the center of a focused photographic subject portiontoward the periphery. Therefore, a boundary between defocusing levelsdoes not appear between the focused photographic subject portion and theother areas, whereby a natural image effect can be ensured in thecaptured image.

In addition, in the generation of the third mask image M3 in thisembodiment, the size of the second mask image M2 is adjusted accordingto the focusing distance or the focal length of the lens section 3.Therefore, when the actual size of a main photographic subject (targetphotographic subject) can be assumed in advance, such as when a mainphotographic subject is the face of a person, a flower, or the like, thesize of an effect reduction area to be set in the third mask image M3can be adjusted to a suitable size without complicated photographicsubject recognition processing.

In the image capture processing of the present embodiment, as a capturedimage, a composite image acquired by a soft focus image and a normaldeveloped image being combined by alpha blending is always recorded.However, in a case where the digital camera 1 has, as a subordinateoperation mode of the imaging mode, a specific imaging mode whereimaging conditions and details of the above-described developmentprocessing are automatically set to be suitable for the image capturingof a specific target, such as a person imaging mode, the control section2 may perform the following image capture processing.

In this case, in image capturing using the soft focus function, thecontrol section 2 generates a soft focus image by the processing at StepSB3, and judges whether the current imaging mode is a specific imagingmode. Then, when the current imaging mode is a specific imaging mode,the control section 2 records, as a captured image, a composite imageacquired by a normal developed image and the soft focus image beingcombined by alpha blending. Conversely, when the current imaging mode isnot a specific imaging mode, the control section 2 performs processingfor recording the generated soft focus image as it is as a capturedimage.

In the above-described case where a composite image acquired bycombining a soft focus image and a normal developed image by alphablending is recorded as a captured image only when the current imagingmode is a specific imaging mode, the actual size of a photographicsubject can be assumed in advance based on the type of the currentimaging mode. Therefore, in this case, the size of an effect reductionarea to be set in the third mask image M3 can be unfailingly adjusted toa size suitable for the actual size of a photographic subject.

Moreover, in the second embodiment as well, the shape of an effectreduction area to be set in the third mask image M3 is oval. However,the shape of this effect reduction area can be changed as necessary, asin the case of the first embodiment.

In addition, when the digital camera 1 has, as subordinate operationmodes of the imaging mode, imaging modes suitable for the imagecapturing of specific targets such as that described above, the shape ofan effect reduction area may be set according to the type of an imagingmode. In this case, as the first mask image M1 representing a basicshape of an effect reduction area, a plurality of first mask imagesrepresenting different basic shapes corresponding to the imaging modesare prepared in advance.

Furthermore, in the first and second embodiments described above, thefirst mask image M1 is stored in the program storage section 7 as maskinformation indicating a basic shape and the like of an effect reductionarea to be set in the third mask image M3, and the second mask image M2is generated from the first mask image M1. However, a configuration maybe adopted in which the second mask image M2 is stored in the programstorage section 7 as mask information indicating a basic shape and thelike of an effect reduction area.

Still further, in the first and second embodiments, when a normaldeveloped image acquired in image capturing and a soft focus imagegenerated from the normal developed image are combined by alphablending, the soft focus image serves as a background. However, in thepresent invention, a configuration may be adopted in which the normaldeveloped image serves as a background. In this configuration, forexample, an image where black and white are reversed is generated as thethird mask image M3.

In either case, the normal developed image and the soft focus image arecombined using the third mask image M3, that is, mask informationregarding an effect reduction area where a defocusing effect by softfocus processing is reduced, whereby a final image is generated. As aresult of this configuration, an intended final image can be easilygenerated by relatively simple image processing as compared to a casewhere a final image is directly generated by the effect being changedfor each pixel of a normal developed image. In particular, in thisconfiguration, since a normal developed image and a soft focus image arecombined by alpha blending, an intended final image can be easilygenerated by exceedingly simple image processing.

Yet still further, in the first and second embodiments, thepredetermined image processing which is performed on an image acquiredin image capturing is soft focus processing for acquiring a defocusingeffect by which a soft atmosphere is created by adding a blurringeffect. However, the present invention can be applied to a case wherethe predetermined image processing is image processing other than thesoft focus processing.

This image processing may be, for example, processing for adding aspecific tone to an original image or adjusting the brightness of anoriginal image, or processing for converting an original image to apainting-style image. Also, the effect of the predetermined imageprocessing is not necessarily given to the entire area of an originalimage and may be given to a specific portion thereof, as with a crossingscreen effect.

Yet still further, in the first and second embodiments, a new compositeimage is generated with a normal developed image acquired at the time ofimage capturing as an original image. However, the original image in thepresent invention may be any image as long as it is an image acquired byimage capturing. For example, it may be a captured image alreadyrecorded on the image storage section 6.

In this case as well, the composite image G3 shown in FIG. 4 can begenerated by processing similar to that of first embodiment. Inaddition, if the focal position, the focal distance, and the like at thetime of the image capturing have been stored in a still image file asadditional information for the captured image, a target photographicsubject therein can be identified by the same method as that of thesecond embodiment.

Yet still further, the present invention can be applied in apparatusesother than the digital camera 1, which include digital photo frames thatare used mainly for displaying an image captured by an arbitrary digitalcamera as an object to be viewed, general-purpose personal computers,and smartphones.

Yet still further, the original image in the present invention may beany image as long as it is an image acquired by image capturing, asdescribed above. For example, it may be a captured image alreadysubjected to arbitrary image processing, regardless of whether thepresent invention has been applied in a digital camera or in otherapparatuses.

While the present invention has been described with reference to thepreferred embodiments, it is intended that the invention be not limitedby any of the details of the description therein but includes all theembodiments which fall within the scope of the appended claims.

What is claimed is:
 1. An image processing apparatus comprising: astorage; and a CPU that is configured to: acquire a position and a sizeof a target photographic subject in an original image acquired by imagecapturing: generate mask information indicating a relation between (i) aposition to be subjected to predetermined image processing in theoriginal image and (ii) a strength of the predetermined imageprocessing, based on the acquired position and the acquired size of thetarget photographic subject; and generate a new image by performingpredetermined image processing of a different strength on differentpositions in the original image by using the generated mask information;wherein: the storage stores basic information of the mask information,the CPU generates the mask information by using the basic informationstored in the storage, the basic information stored in the storageincludes a basic shape of a processing area that is subjected to thepredetermined image processing, and the basic information stored in thestorage includes information regarding a change in the strength of thepredetermined image processing in the processing area that is subjectedto the predetermined image processing, the information regarding thechange in the strength indicating that the strength is changed to begradually increased from a center side to a periphery side.
 2. An imageprocessing apparatus comprising: a CPU that is configured to: acquire aposition and a size of a target photographic subject in an originalimage acquired by image capturing; generate mask information indicatinga relation between (i) a position to be subjected to predetermined imageprocessing in the original image and (ii) a strength of thepredetermined image processing, based on the acquired position and theacquired size of the target photographic subject; and generate a newimage by performing predetermined image processing of a differentstrength on different positions in the original image by using thegenerated mask information; wherein: the CPU generates the new image bycombining the original image and a processed image acquired by thepredetermined image processing being performed on the original image, byusing the generated mask information, and the CPU generates, as thegenerated mask information, information indicating differenttransmissivities for different positions in the original image, andgenerates the new image by (i) setting, using the generated maskinformation, positions of one of the processed image and the originalimage to corresponding transmissivities indicated by the maskinformation and (ii) combining one of the processed image and theoriginal image with the other one of the processed image and theoriginal image.
 3. An image processing method comprising: acquiring aposition and a size of a target photographic subject in an originalimage acquired by image capturing; generating mask informationindicating a relation between (i) a position to be subjected topredetermined image processing in the original image and (ii) a strengthof the predetermined image processing, based on the acquired positionand the acquired size of the target photographic subject; and generatinga new image by performing predetermined image processing of a differentstrength on different positions in the original image by using thegenerated mask information, wherein the generating the mask informationgenerates the mask information by using information regarding a basicshape of a processing area that is subjected to the predetermined imageprocessing and information regarding a change in the strength of thepredetermined image processing in the processing area that is subjectedto the predetermined image processing, the information regarding thechange in the strength indicating that the strength is changed to begradually increased from a center side to a periphery side.
 4. The imageprocessing method according to claim 3, wherein the image capturingwhich acquires the original image from which the acquiring acquires theposition and the size of the target photographic subject is performed byan imaging section.
 5. The image processing method according to claim 3,further comprising identifying, as the target photographic subject, aphotographic subject for which focal adjustment has been made and whichis shown in the original image acquired by image capturing, wherein theacquiring acquires the position and the size of the target photographicsubject in the original image, based on focal information of theidentified photographic subject.
 6. An image processing methodcomprising: acquiring a position and a size of a target photographicsubject in an original image acquired by image capturing; generatingmask information indicating a relation between (i) a position to besubjected to predetermined image processing in the original image and(ii) a strength of the predetermined image processing, based on theacquired position and the acquired size of the target photographicsubject; and generating a new image by performing predetermined imageprocessing of a different strength on different positions in theoriginal image by using the generated mask information, wherein: thegenerating the mask information generates, as the generated maskinformation, information indicating different transmissivities fordifferent positions in the original image, and the generating the newimage generates the new image by (i) setting, using the generated maskinformation, positions of one of the processed image and the originalimage to corresponding transmissivities indicated by the generated maskinformation and (ii) combining one of the processed image and theoriginal image with the other one of the processed image and theoriginal image.
 7. The image processing method according to claim 6,wherein the image capturing which acquires the original image from whichthe acquiring acquires the position and the size of the targetphotographic subject is performed by an imaging section.
 8. The imageprocessing method according to claim 6, further comprising identifying,as the target photographic subject, a photographic subject for whichfocal adjustment has been made and which is shown in the original imageacquired by image capturing, wherein the acquiring acquires the positionand the size of the target photographic subject in the original image,based on focal information of the identified photographic subject.
 9. Animage processing method comprising: detecting a target photographicsubject in an original image acquired by image capturing, as an areawith a predetermined shape in the original image; acquiring a positionand a size of the detected target photographic subject area; generatinga mask image indicating a relation between (i) a position to besubjected to predetermined image processing in the original image and(ii) a strength of the predetermined image processing, based on theacquired position and the acquired size of the detected targetphotographic subject area; and generating a new image by performingpredetermined image processing of a different strength on differentpositions in the original image by using the generated mask image,wherein the generating the mask image generates the mask image centeringon a position shifted from a center of the detected target photographicsubject area.
 10. The image processing method according to claim 9,wherein the image capturing which acquires the original image from whichthe acquiring acquires the position and the size of the targetphotographic subject is performed by an imaging section.
 11. An imageprocessing method comprising: acquiring a position and a size of atarget photographic subject in an original image acquired by imagecapturing; generating mask information indicating a relation between (i)a position to be subjected to predetermined image processing in theoriginal image and (ii) a strength of the predetermined imageprocessing, based on the acquired position and the acquired size of thetarget photographic subject; and generating a new image by performingpredetermined image processing of a different strength on differentpositions in the original image by using the generated mask information,wherein: the acquiring further acquires a rotation direction of thetarget photographic subject in the original image, and the generatingthe mask information generates the mask information in furtherconsideration of the acquired rotation direction of the targetphotographic subject.
 12. The image processing method according to claim11, wherein the image capturing which acquires the original image fromwhich the acquiring acquires the position and the size of the targetphotographic subject is performed by an imaging section.